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Tennessee  State  University 
Engineering  Research  Institute 
College  of  Engineering,  Technology  and  Computer  Science 

Final  Report:  Enhancement  of  Robotics  Laboratory 

Tennessee  State  University  received  a  grant  of  $179,625  from  the  Office  of  Naval 
Research  under  FY  2002  DoD  Infrastructure  support  programs  for  BHCU/MI  BAA  No: 
DAAD19-02-R-002.  The  grant  provided  funds  to  enhance  the  existing  mobile  robotics 
laboratory  through  the  purchase  of  five  All  Terrain  Robotic  Vehicles  (AVTR),  which  will 
be  used  indoors  and  outdoors.  The  existing  B14  mobile  robot  at  that  time  was  about  six 
years  old  and  had  run  down  its  useful  life.  In  addition  the  software  was  not  current  and  it 
did  not  support  the  latest  software.  Other  small  mobile  robots  (Trilibots)  had  many  on 
board  sensors,  but  they  had  limited  capability  and  they  could  not  go  straight  and  those 
robots  including  B14  could  not  be  used  outside.  However,  they  were  good  for  student 
training,  but  not  for  quality  research. 

It  was  also  proposed  that  the  acquisition  of  new  robots  would  be  a  natural  extension  of 
research  activities  of  the  Center  of  Neural  Networks-funded  by  ONR  since  1992  to  test, 
validate  and  develop  robotics  technologies  in  indoor  and  outdoor  type  of  environments. 
This  would  expand  the  level  of  expertise  of  research  faculty  at  TSU  as  well  as  help 
students  involved  in  research  activities  to  be  exposed  to  the  state-of-the-art  equipment 
through  senior  capstone  design  projects  for  undergraduates  and  masters  theses  for 
graduate  students.  The  research  will  also  enrich  the  curriculum  in  appropriate  courses. 

It  was  assumed  that  the  research  activities  will  benefit  with  the  acquisition  of  robots,  and 
the  proposed  equipment  will  help  TSU  accomplish  its  mission  and  the  mission  of  DoD  to 
meet  the  intended  goals  and  objectives  by  having  access  to  the-state-of-the-art  equipment 
and  test  technologies  in  the  area  of  mobile  robotics,  distributed  computing,  intelligent 
multi-agents  and  sensory  fusion,  wireless  communication  and  integration.  It  was  also 
expected  that  such  research  activities  will  contribute  to  the  advances  of  the  literature  in 
the  areas  of  mobile  robots  and  its  applications  in  solving  real-world  problems  in  both 
commercial  as  well  as  government  type  applications. 

The  overall  goal  of  the  enhancement  of  robotics  research  laboratory  is  to  conduct 
research  in  cooperative  behavior  of  multi-agent  autonomous  robots.  It  is  envisioned  that  a 
group  of  cooperative  robots  will  operate  in  combined  operator/automated  or  autonomous 
modes  with  sufficient  intelligence  to  negotiate  obstacles  while  maintaining  constant  line- 
of-sight  communication  with  one-another  in  the  areas  of  security  and  surveillance.  Local 
software  will  provide  knowledge-based  feedback/recommendations  on  emergency 
operational  procedures  and  alternatives  to  operators  and  others.  Wireless  communications 
will  be  implemented  using  advanced  off-the  shelf  network  components  facilitating  ease 
of  maintenance,  integration,  expandability  to  other  existing  or  future  support  systems. 
Robot  wireless  network  will  act  as  wireless  outside  the  building  for  other  applications 
and  systems,  such  as  first  responder  communications  or  tracking. 
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The  report  highlights  our  accomplishments  in  the  areas  of  image  processing,  multiple 
robot  coordination,  and  human-robot  interaction.  The  report  also  lists  the  impact  of  robots 
on  curriculum  and  minority  students  in  science,  engineering  and  mathematics  (SEM) 
areas  through  undergraduate  capstone  senior  design  projects  and  masters  theses 
completed  and  masters  theses  in  progress,  and  the  faculty  publications. 

Accomplishments 

We  have  purchased  three  Pioneers  3-AT  robots,  two  Pioneer  3-DX  robots  without 
internal  PC,  and  two  Pioneer  3-DX  robots  with  internal  PC.  Totally  we  have  nine  (9) 
Pioneer  mobile  robots.  With  these  robots,  we  have  explored  the  following  research  areas: 

Image  Processing 

Each  robot  is  equipped  with  a  computer  vision  system  and  we  have  implemented  several 
image  processing/  computer  vision  algorithms  for  robot  navigation  and  human-robot 
interaction.  Two  different  vision  systems  are  used  on  the  robots:  Bumblebee  Stereo 
Vision  System  and  pan-tilt-zoom  video  cameras.  On  one  Pioneer3  robot,  a  Bumblebee 
Stereo  Camera  Vision  System  is  installed  on  a  Directed  Perception  pan-tilt  unit,  as  shown 
in  Figure  1 .  Several  software  components  were  written  for  this  vision  system,  including 
camera  DLL  (dynamic  link  library),  pan-tilt  unit  DLL,  and  Display  ActiveX  control. 


Figure  1 :  The  Bumblebee  stereo  camera  and  the  Directed  Perception  pan-tilt  unit 

A  color-based  object  detection  and  recognition  algorithm  was  implemented  on  this  vision 
system.  The  robot  detects  three  types  of  plastic  bottles:  Coke  Cola,  Mellow  Yellow,  and 
Aquifina.  Visual  servoing  behavior  to  make  the  robot  pick  up  the  bottle  was  also 
implemented,  including  saccade  and  VOR  (vestibulo-ocular  reflex).  A  Sony  or  Canon 
pan-tilt-zoom  video  conference  camera  is  mounted  on  the  top  or  the  front  of  some  robots. 
A  color-based  bottle  detection  and  visual  servoing  was  implemented,  and  a  picking-up- 
the-lying-bottle-using-the-arm  behavior  was  accomplished  using  the  vision  system. 
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Multiple  Robot  Coordination 

We  have  two  different  types  of  robots,  Pioneer3  AT  and  Pioneer3  DX,  each  has  different 
sensing  and  action  capabilities.  We  explored  multiple  heterogeneous  robot  coordination 
using  two  different  robots  to  implement  a  search-and-retrieve  task.  Figure  2  shows  one 
robot,  Pioneer3  watching  another  robot,  DX,  picking  up  a  lying  bottle  from  the  floor. 
Pioneer3has  a  stereo  camera  mounted  on  a  pan-tilt  unit  and  a  gripper.  The  gripper  can 
grasp  a  standing  bottle  and  carry  it  to  the  home  position,  however  it  cannot  pick  up  a 
bottle  if  the  bottle  is  lying  on  the  floor.  The  other  robot,  DX,  has  an  arm  and  a  camera. 
The  camera  is  mounted  very  close  to  the  ground  level  due  to  mechanical  constrains,  and 
it  can  tilt  up  or  down.  The  arm  has  6  degrees-of-ffeedom  (DOF),  and  each  joint  is  driven 
by  a  hobby  servo  motor.  These  hobby  servo  motors  are  imprecise,  and  the  servo  motor 
controller  only  supports  independent  joint  position  control,  i.e.,  coordinated  joint 
movement  is  not  possible.  Thus,  although  one  can  use  this  arm  to  pick  up  a  bottle  that  is 
lying  on  the  floor,  the  picking-up  movement  sequence  has  to  be  pre-determined  by 
experiments,  and  is  performed  with  a  fixed  relative  pose  between  the  robot  and  the  bottle. 
Also,  one  cannot  use  the  arm  to  carry  the  bottle  to  the  base  because  the  servo  motors  are 
weak,  instead,  Pioneer3  is  required  to  carry  the  bottle.  Several  visual  servoing  loops  and 
coordination  behaviors  were  implemented  to  accomplish  this  search-and-retrieve  task. 


Figure  2:  Two  Pioneer  3  robots  DX(left)  and  AT  (right)  cooperate  to  pick  up  a  lying 
bottle 

In  our  initial  implementation,  the  subtask  decomposition  and  allocation  is  pre-determined 
by  the  designer.  The  search-and-retrieve  mission  is  carried  out  in  the  following  fashion. 
First,  both  Pioneer3and  DX  robots  stay  at  the  home  base.  Given  a  map,  Pioneer3was 
commanded  to  move  into  an  area  that  contains  different  plastic  bottles  (Coke,  Mellow 
Yellow,  and  Aquafina).  Pioneer3  scans  the  environment  and  reports  the  direction  of  all 
the  bottles  that  it  has  detected  to  the  user.  The  user  then  commands  the  robot  to  retrieve  a 
certain  bottle,  say  Coke.  Pioneer  then  moves  close  to  the  Coke  bottle  and  detects  the 
bottle’s  pose.  If  the  bottle  is  standing  on  the  floor,  then  Pioneer3aligns  its  gripper  with 
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the  bottle,  grasps  it,  returns  to  the  base  following  the  path  that  it  came  in,  and  finally 
releases  the  bottle.  The  user  then  asks  the  robot  to  retrieve  another  bottle,  say  Aquafina. 
Pioneer3  moves  back  to  the  area  and  moves  close  to  the  Aquafina  bottle  because  it 
memorizes  the  bottle’s  direction  from  last  environment  scanning.  Again,  it  detects  the 
bottle’s  pose.  Suppose  the  bottle  is  lying  on  the  floor  and  Pioneer3cannot  pick  it  up.  It 
then  activates  DX,  and  sends  the  bottle’s  pose  information  to  DX.  DX  moves  to  a 
suitable  location  under  Pioneer3s  command,  then  picks  up  the  bottle  using  the  arm. 
Pioneer3then  aligns  its  gripper  with  the  bottle,  grasp  it,  returns  to  the  base,  and  finally 
releases  the  bottle. 

Modeling  Human-Robot  Interaction 

We  implemented  a  human-robot  interaction  model  on  a  Pioneer3  AT  robot.  This  human- 
robot  model  has  core  functionalities  that  provide  a  robot  with  perception,  awareness  and 
social  appropriateness.  A  multiple  agent  system  was  utilized,  as  shown  in  Figure  3.  It 
includes  Human  Input  Agent,  Human  Agent,  Capability  Agent,  and  a  Human  Database. 
The  Human  Agent  gets  input  from  a  Human  Input  Agents  and  operates  on  a  Human 
Database,  and  itself  is  composed  of  Monitoring  Agent  and  Interaction  Agent.  The 
monitor  agent  monitors  the  environment  for  features  or  events  that  indicate  that  people 
are  present  or  active.  Currently  the  monitor  can  receive  inputs  from  various  detectors 
such  as  visual  (face  detection)  or  environmental  (motion).  The  Interaction  Agent 
coordinates  the  robot’s  role  in  the  interaction.  It  processes  input  and  the  knowledge  of 
the  situation  to  determine  the  intention  of  the  people  in  the  environment  and  determine 
the  appropriate  response. 


Figure  3 :  Architecture  for  human-robot  interaction 

The  system  has  been  implemented  on  a  Pioneer  3  robot  (cf.  Figure  1  and  2),  which  has  a 
stereo  camera  and  a  gripper.  The  context  for  our  implementation  is  the  interaction 
between  a  person  and  a  robot  that  can  perform  helper  tasks  for  the  person.  The  robot 
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2.  Basel  Mohamoud,  “Development  of  intelligent  behaviors  for  multiple  robots 
cooperative  control.” 

3  Green  Sheldon,  “Rout  planning  and  navigation  with  landmark  recognition  and 
localization. 

4.  Swapana  Pelle,  “Human  guiding  robot.” 

Currently  we  have  robotics  laboratory  in  electrical  and  mechanical  engineering,  and  plans 
are  to  develop  another  laboratory  in  computer  science  department 

Faculty  Publications 

Tamara  E.  Rogers,  Jian  Peng,  and  Saleh  Zein-Sabatto,  “Modeling  human-robot 
interaction  for  intelligent  mobile  robotics”,  submitted  to  1 4th  IEEE  International 
Workshop  on  Robot  and  Human  Interactive  Communication ,  Nashville,  TN,  August  13- 
15,  2005 

Jian  Peng,  Saleh  Zein-Sabatto,  and  Ali  Sekmen,  “Implementing  search-and-retrieve  tasks 
by  multiple  heterogeneous  robots”,  submitted  to  IASTED  International  Conference  on 
Robotics  And  Applications,  Cambridge,  MA,  October  31  -  November  2 , 2005 

Impact  of  Robots  on  undergraduate  minority  students  in  SGM 

As  a  result  of  addition  of  new  robots  the  interest  of  students  has  increased  among 
electrical,  mechanical  and  computer  science  students,  and  an  interdisciplinary 
undergraduate  research  group  in  robotics  is  formed,  hopefully  it  will  lead  to 
interdisciplinary  research  projects  in  robotics.  The  numbers  of  senior  projects  and  masters 
theses  have  increased,  and  hopefully  the  research  will  generate  Ph  D.  dissertations. 
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